Artificial intelligence-assisted double reading of chest radiographs to detect clinically relevant missed findings: a two-centre evaluation.

OBJECTIVES: To evaluate an artificial intelligence (AI)-assisted double reading system for detecting clinically relevant missed findings on routinely reported chest radiographs. METHODS: A retrospective study was performed in two institutions, a secondary care hospital and tertiary referral oncology centre. Commercially available AI software performed a comparative analysis of chest radiographs and radiologists' authorised reports using a deep learning and natural language processing algorithm, respectively. The AI-detected discrepant findings between images and reports were assessed for clinical relevance by an external radiologist, as part of the commercial service provided by the AI vendor. The selected missed findings were subsequently returned to the institution's radiologist for final review. RESULTS: In total, 25,104 chest radiographs of 21,039 patients (mean age 61.1 years ± 16.2 [SD]; 10,436 men) were included. The AI software detected discrepancies between imaging and reports in 21.1% (5289 of 25,104). After review by the external radiologist, 0.9% (47 of 5289) of cases were deemed to contain clinically relevant missed findings. The institution's radiologists confirmed 35 of 47 missed findings (74.5%) as clinically relevant (0.1% of all cases). Missed findings consisted of lung nodules (71.4%, 25 of 35), pneumothoraces (17.1%, 6 of 35) and consolidations (11.4%, 4 of 35). CONCLUSION: The AI-assisted double reading system was able to identify missed findings on chest radiographs after report authorisation. The approach required an external radiologist to review the AI-detected discrepancies. The number of clinically relevant missed findings by radiologists was very low. CLINICAL RELEVANCE STATEMENT: The AI-assisted double reader workflow was shown to detect diagnostic errors and could be applied as a quality assurance tool. Although clinically relevant missed findings were rare, there is potential impact given the common use of chest radiography. KEY POINTS: • A commercially available double reading system supported by artificial intelligence was evaluated to detect reporting errors in chest radiographs (n=25,104) from two institutions. • Clinically relevant missed findings were found in 0.1% of chest radiographs and consisted of unreported lung nodules, pneumothoraces and consolidations. • Applying AI software as a secondary reader after report authorisation can assist in reducing diagnostic errors without interrupting the radiologist's reading workflow. However, the number of AI-detected discrepancies was considerable and required review by a radiologist to assess their relevance.

Artificial Intelligence Tool for Detection and Worklist Prioritization Reduces Time to Diagnosis of Incidental Pulmonary Embolism at CT.

Purpose: To evaluate the diagnostic efficacy of artificial intelligence (AI) software in detecting incidental pulmonary embolism (IPE) at CT and shorten the time to diagnosis with use of radiologist reading worklist prioritization. Materials and Methods: In this study with historical controls and prospective evaluation, regulatory-cleared AI software was evaluated to prioritize IPE on routine chest CT scans with intravenous contrast agent in adult oncology patients. Diagnostic accuracy metrics were calculated, and temporal end points, including detection and notification times (DNTs), were assessed during three time periods (April 2019 to September 2020): routine workflow without AI, human triage without AI, and worklist prioritization with AI. Results: In total, 11 736 CT scans in 6447 oncology patients (mean age, 63 years ± 12 [SD]; 3367 men) were included. Prevalence of IPE was 1.3% (51 of 3837 scans), 1.4% (54 of 3920 scans), and 1.0% (38 of 3979 scans) for the respective time periods. The AI software detected 131 true-positive, 12 false-negative, 31 false-positive, and 11 559 true-negative results, achieving 91.6% sensitivity, 99.7% specificity, 99.9% negative predictive value, and 80.9% positive predictive value. During prospective evaluation, AI-based worklist prioritization reduced the median DNT for IPE-positive examinations to 87 minutes (vs routine workflow of 7714 minutes and human triage of 4973 minutes). Radiologists' missed rate of IPE was significantly reduced from 44.8% (47 of 105 scans) without AI to 2.6% (one of 38 scans) when assisted by the AI tool (P < .001). Conclusion: AI-assisted workflow prioritization of IPE on routine CT scans in oncology patients showed high diagnostic accuracy and significantly shortened the time to diagnosis in a setting with a backlog of examinations.Keywords: CT, Computer Applications, Detection, Diagnosis, Embolism, Thorax, ThrombosisSupplemental material is available for this article.© RSNA, 2023See also the commentary by Elicker in this issue.

A deep learning-based application for COVID-19 diagnosis on CT: The Imaging COVID-19 AI initiative.

BACKGROUND: Recently, artificial intelligence (AI)-based applications for chest imaging have emerged as potential tools to assist clinicians in the diagnosis and management of patients with coronavirus disease 2019 (COVID-19). OBJECTIVES: To develop a deep learning-based clinical decision support system for automatic diagnosis of COVID-19 on chest CT scans. Secondarily, to develop a complementary segmentation tool to assess the extent of lung involvement and measure disease severity. METHODS: The Imaging COVID-19 AI initiative was formed to conduct a retrospective multicentre cohort study including 20 institutions from seven different European countries. Patients with suspected or known COVID-19 who underwent a chest CT were included. The dataset was split on the institution-level to allow external evaluation. Data annotation was performed by 34 radiologists/radiology residents and included quality control measures. A multi-class classification model was created using a custom 3D convolutional neural network. For the segmentation task, a UNET-like architecture with a backbone Residual Network (ResNet-34) was selected. RESULTS: A total of 2,802 CT scans were included (2,667 unique patients, mean [standard deviation] age = 64.6 [16.2] years, male/female ratio 1.3:1). The distribution of classes (COVID-19/Other type of pulmonary infection/No imaging signs of infection) was 1,490 (53.2%), 402 (14.3%), and 910 (32.5%), respectively. On the external test dataset, the diagnostic multiclassification model yielded high micro-average and macro-average AUC values (0.93 and 0.91, respectively). The model provided the likelihood of COVID-19 vs other cases with a sensitivity of 87% and a specificity of 94%. The segmentation performance was moderate with Dice similarity coefficient (DSC) of 0.59. An imaging analysis pipeline was developed that returned a quantitative report to the user. CONCLUSION: We developed a deep learning-based clinical decision support system that could become an efficient concurrent reading tool to assist clinicians, utilising a newly created European dataset including more than 2,800 CT scans.

The RSNA International COVID-19 Open Radiology Database (RICORD).

The coronavirus disease 2019 (COVID-19) pandemic is a global health care emergency. Although reverse-transcription polymerase chain reaction testing is the reference standard method to identify patients with COVID-19 infection, chest radiography and CT play a vital role in the detection and management of these patients. Prediction models for COVID-19 imaging are rapidly being developed to support medical decision making. However, inadequate availability of a diverse annotated data set has limited the performance and generalizability of existing models. To address this unmet need, the RSNA and Society of Thoracic Radiology collaborated to develop the RSNA International COVID-19 Open Radiology Database (RICORD). This database is the first multi-institutional, multinational, expert-annotated COVID-19 imaging data set. It is made freely available to the machine learning community as a research and educational resource for COVID-19 chest imaging. Pixel-level volumetric segmentation with clinical annotations was performed by thoracic radiology subspecialists for all COVID-19-positive thoracic CT scans. The labeling schema was coordinated with other international consensus panels and COVID-19 data annotation efforts, the European Society of Medical Imaging Informatics, the American College of Radiology, and the American Association of Physicists in Medicine. Study-level COVID-19 classification labels for chest radiographs were annotated by three radiologists, with majority vote adjudication by board-certified radiologists. RICORD consists of 240 thoracic CT scans and 1000 chest radiographs contributed from four international sites. It is anticipated that RICORD will ideally lead to prediction models that can demonstrate sustained performance across populations and health care systems.

Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors.

OBJECTIVE: The objective was to identify barriers and facilitators to the implementation of artificial intelligence (AI) applications in clinical radiology in The Netherlands. MATERIALS AND METHODS: Using an embedded multiple case study, an exploratory, qualitative research design was followed. Data collection consisted of 24 semi-structured interviews from seven Dutch hospitals. The analysis of barriers and facilitators was guided by the recently published Non-adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework for new medical technologies in healthcare organizations. RESULTS: Among the most important facilitating factors for implementation were the following: (i) pressure for cost containment in the Dutch healthcare system, (ii) high expectations of AI's potential added value, (iii) presence of hospital-wide innovation strategies, and (iv) presence of a "local champion." Among the most prominent hindering factors were the following: (i) inconsistent technical performance of AI applications, (ii) unstructured implementation processes, (iii) uncertain added value for clinical practice of AI applications, and (iv) large variance in acceptance and trust of direct (the radiologists) and indirect (the referring clinicians) adopters. CONCLUSION: In order for AI applications to contribute to the improvement of the quality and efficiency of clinical radiology, implementation processes need to be carried out in a structured manner, thereby providing evidence on the clinical added value of AI applications. KEY POINTS: • Successful implementation of AI in radiology requires collaboration between radiologists and referring clinicians. • Implementation of AI in radiology is facilitated by the presence of a local champion. • Evidence on the clinical added value of AI in radiology is needed for successful implementation.

Radiologists' Usage of Social Media: Results of the RANSOM Survey.

The growing use of social media is transforming the way health care professionals (HCPs) are communicating. In this changing environment, it could be useful to outline the usage of social media by radiologists in all its facets and on an international level. The main objective of the RANSOM survey was to investigate how radiologists are using social media and what is their attitude towards them. The second goal was to discern differences in tendencies among American and European radiologists. An international survey was launched on SurveyMonkey ( https://www.surveymonkey.com ) asking questions about the platforms they prefer, about the advantages, disadvantages, and risks, and about the main incentives and barriers to use social media. A total of 477 radiologists participated in the survey, of which 277 from Europe and 127 from North America. The results show that 85 % of all survey participants are using social media, mostly for a mixture of private and professional reasons. Facebook is the most popular platform for general purposes, whereas LinkedIn and Twitter are more popular for professional usage. The most important reason for not using social media is an unwillingness to mix private and professional matters. Eighty-two percent of all participants are aware of the educational opportunities offered by social media. The survey results underline the need to increase radiologists' skills in using social media efficiently and safely. There is also a need to create clear guidelines regarding the online and social media presence of radiologists to maximize the potential benefits of engaging with social media.

The Impact of Information Technology on Radiology Services: An Overview: Article based on PhD dissertation to obtain the degree of doctor in medical sciences, defended at the University of Antwerp on July 14, 2016.

The main objective of this paper is to provide an overview of the impact of information technology on radiology services during the past 15 years and to promote awareness of the digital revolution that is taking place in health care, including radiology. The combination of two major innovations is playing a central role in this revolution, namely, the Internet and the digitisation of medical information. The various stages of the Internet development and their relationship with the almost simultaneously ongoing digitisation of the radiology department are described. The onset of teleradiology services and the more recent trend toward the usage of cloud-based networks and services are explained. The recent changes in digital communication and electronic transmission of medical information are discussed, hereby paying attention to the value of social media in medicine and radiology in particular. Finally, the future prospects of health care and medical imaging are outlined in the spotlight of today's major trends, and the role of the radiologist in this quickly changing environment is redefined.

Social media for radiologists: an introduction.

UNLABELLED: Social media, which can be defined as dynamic and interactive online communication forums, are becoming increasingly popular, not only for the general public but also for radiologists. In addition to assisting radiologists in finding useful profession-related information and interactive educational material in all kinds of formats, they can also contribute towards improving communication with peers, clinicians, and patients. The growing use of social networking in healthcare also has an impact on the visibility and engagement of radiologists in the online virtual community. Although many radiologists are already using social media, a large number of our colleagues are still unaware of the wide spectrum of useful information and interaction available via social media and of the added value these platforms can bring to daily practice. For many, the risk of mixing professional and private data by using social media creates a feeling of insecurity, which still keeps radiologists from using them. In this overview we aim to provide information on the potential benefits, challenges, and inherent risks of social media for radiologists. We will provide a summary of the different types of social media that can be of value for radiologists, including useful tips on how to use them safely and efficiently. MAIN MESSAGES: • Online social networking enhances communication and collaboration between peers • Social media facilitate access to educational and scientific information • Recommendations and guidelines from policymakers and professional organisations are needed • Applications are desired for efficient and secure exchange of medical images in social media.

Comparison of European (ESR) and American (ACR) white papers on teleradiology: patient primacy is paramount.

The ACR and European Society of Radiology white papers on teleradiology propose best practice guidelines for teleradiology, with each body focusing on its respective local situation, market, and legal regulations. The organizations have common viewpoints, the most important being patient primacy, maintenance of quality, and the "supplementary" position of teleradiology to local services. The major differences between the white papers are related mainly to the market situation, the use of teleradiology, teleradiologist credentialing and certification, the principles of "international" teleradiology, and the need to obtain "informed consent" from patients. The authors describe these similarities and differences by highlighting the background and context of teleradiology in Europe and the United States.